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DETAILED ACTION 



Response to Amendment 

1 . Applicants' response to the last Office Action, filed on November 25 th , 2008 has 
been entered and made of record. 
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Response to Arguments 

2. Applicant's arguments filed November 25 th , 2008 have been fully considered but 

they are not persuasive. 

a. In response to applicant's remark that a change detector cannot be used 
with a moving camera, because the background is not static and thus VM is not 
combinable with Laumeyer. The question of obviousness is resolved on the 
basis of underlying factual determinations including (1) the scope and content of 
the prior art, (2) any differences between the claimed subject matter and the prior 
art, and (3) the level of skill in the art. Graham v. John Deere Co., 383 U.S. 1 , 
17-18 (1966). The combination of familiar elements according to known methods 
is likely to be obvious when it does no more than yield predictable results. KSR 
Int'l Co. v. Teleflex, Inc., 127 S. Ct. 1727, 1739 (2007). Modifying a stationary 
camera that classify objects and uses a background subtraction method 
as disclosed VM, to a moving camera that identify objects and identification 
includes separating objects of interest from the background features that are 
non-important (Paragraph [0045], lines 26-28) as disclosed by Laumeyer is the 
combination of familiar elements according to known methods. This combination 
is obvious because it does no more than yield the predictable results. It is 
unconvincing that this combination is "uniquely challenging or difficult for one of 
ordinary skill in the art," (see Leapfrog Enters., Inc. v. Fisher-Price, Inc., 485 F.3d 
1157, 1162 (Fed. Cir. 2007) (citing KSR, 127 S. Ct. at 1741)), and applicant have 
provided no evidence to the contrary. 



Application/Control Number: 10/805,946 Page 4 

Art Unit: 2624 

b. In response to applicant's remark that "VM does not address detecting 
object components of different sizes, using classifiers for detecting components 
at multiple scales." It is noted that VM does teach detecting object components 
of different sizes. VM clearly shows the matching of an observed size of a 
mobile object with a previously gathered size of the object in order to classify the 
object observed. This was clearly shown in the office action; please see action 
for detail citations. 

c. In response to applicant's remark that there is nothing in VM's teachings 
regarding factor ranges that implies accumulating confidence score across 
multiple frames. It is noted that VM teaches that a confidence factor for each 
classification, which ranges from 0 to 1 . Examiner reads confidence factor to be 
the same as confidence score. 

d. In response to applicant's remark that Laumeyer does not disclose the use 
of classifiers to identify signs in video stream. It is noted that Examiner does not 
depend on Laumeyer to teach classifiers but depends on VM to teach classifiers. 
Examiner uses Laumeyer for the teachings of a moving camera. Please see 
analysis of VM for identification of classifiers. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1 -29, 33-35, 37-42, 45, 47, 49-51 , 53, 55, 58-80, 84-85, 87-92, and 1 05 
are rejected under 35 U.S.C. 1 03(a) as being unpatentable over VM in view of 
Laumeyer et al. (US 2007/0154067 hereinafter "Laumeyer"). 

As to claim 1 , VM teaches method for detecting one or more objects belonging to 
the same object class comprising the steps of: a), receiving a video sequence from a 
video camera (each camera is attached to a processing unit using a video capture card 
that hosts a set of image video object kernel; page 457, left-hand column, lines 25-28) 
comprised of a plurality of image frames (all video streams consists of frames); b) 
applying one or more classifiers to detect components of objects in an image frame in 
the video sequence (each scene may have one of a number of classifications; page 
459, left-hand column, lines 11-14), wherein the component classifiers include 
classifiers for detecting components at multiple scales (classification methods are based 
on matching of the observed size of the mobile object with previously gathered 
information of typical sizes for object of all classes; page 459, left-hand column, lines 
26-29); c) computing a confidence score based in part on the response from the one or 
more component detectors (each classification has a confidence factor between 0 and 
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1 , 1 being maximum confidence and 0 means no confidence; page 459, left-hand 
column, lines 15-17); d) repeating steps b) and c) to detect components of objects 
belonging to the same object class in additional images frames in the video sequence 
(page 459, left-hand column, lines 8-28); and e) accumulating confidence scores from 
the component detectors to determine if an object is detected (page 459, left-hand 
column, lines 8-37); However, VM does not teach and wherein said method is adapted 
for detecting moving and stationary objects from a moving or stationary video camera. 

Laumeyer teaches a method for identifying objects depicted in a video stream 
(Title) wherein the cameras collecting the video data is mounted on a vehicle (Fig. 3B) 
and the objects being detected are road signs (Paragraph [0059], lines 1-2) and 
pedestrian (Paragraph [0059], lines 22-24). Thus Laumeyer's vehicle mounted cameras 
that detect road signs and pedestrian reads on the claimed moving video camera that's 
capable of detecting moving and stationary objects. Therefore, it would have been 
obvious for one ordinarily skilled in the art at the time the invention was made to 
combine VM's traffic monitoring method with Laumeyer's vehicle cameras in order to 
allow individual vehicles to monitor traffic outside their vehicles. 
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As to claim 2, VM teaches wherein if accumulated confidence scores indicate 
high confidence of a presence of an object, the method further comprising the step of: 
identifying the detected components to be an object of a particular type (page 459, left- 
hand column lines 22-25). 

As to claim 3, VM teaches wherein the object class is a vehicle (page 459, left- 
hand column lines 12-14). 

As to claim 4, VM teaches if an object is detected, outputting a detection signal 
and object position (page 459, left-hand column lines 43-58). 

As to claim 5, VM teaches testing geometry constraints on a spatial arrangement 
of detected components in an image; and applying whole-appearance classifiers an 
image patch that contains the detected components and which is aligned according to 
the position of the detected components (page 459, left-hand column lines 26-29). 

As to claim 6, VM teaches wherein the geometry constraints are derived from 
camera parameters (page 459, left-hand column lines 20-21). 

As to claim 7, VM teaches wherein the geometry constraints are derived from 
object size (page 459, left-hand column line 27). 
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As to claim 8, VM teaches wherein the geometry constraints are derived from a 
location of an object appearance in the image (page 459, left-hand column line 46). 

As to claim 9, VM teaches wherein the whole appearance classifiers detect entire 
or partial object appearance, the entire or partial object appearance being aligned 
according to positioning of at least two components (page 459, left-hand column lines 
26-29). 

As to claim 10, VM teaches wherein the component classifiers include classifiers 
for detecting components at multiple scales (page 459, left-hand column lines 30-36). 

As to claim 1 1 , VM teaches wherein component classifiers are defined by 
discriminant features and decision rules which are learned through boosted training 
(page 459, left-hand column lines 37-38). 

As to claim 12, VM teaches wherein the discriminant features include corners 
(page 461 , fig. 4 shows an object that's been identified and the system forming a box 
around the object, the box has corners). 

As to claim 13, VM teaches wherein the discriminant features include horizontal 
edges (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has horizontal edges). 
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As to claim 14, VM teaches wherein the discriminant features include vertical 
edges (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has vertical edges). 

As to claim 15, VM teaches wherein the discriminant features include horizontal 
stripes (page 461, fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain horizontal stripes). 

As to claim 16, VM teaches wherein the discriminant features include vertical 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain vertical stripes). 

As to claim 17, VM teaches wherein the discriminant features include diagonal 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain diagonal stripes). 

As to claim 18, Laumeyer teaches performing an online adaptation to adapt a 
classifier structure to an online pattern (the method of rapidly identifying, locating, and 
storing images of objects from the frames maybe implemented using the internet; 
[0012], lines 1-6, 9-10). 
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As to claim 19, Laumeyer teaches wherein the step of performing an online 
adaptation further comprises the step of: applying a dynamic switching strategy to direct 
the detector to take appropriate weak classifiers as discriminants according to auxiliary 
information about the online pattern (selected portions of individual frames are used to 
confirm that a set of several images in fact represents a single object of a class of 
objects; [0012], lines 11-24). 

As to claim 20, VM teaches wherein the one or more classifiers include 
overlapping component classifiers (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object; this box is formed by first 
gathering the structure of the object being classified then the corresponding shape is 
matched with the 2D mask to yield a scaled information. From the 3D points 
information such as size, position and speed maybe extracted (Page 462 left-hand 
column lines 3-8). The overlapping component may be the different information 
extracted from the 3D information, since all information is based on a single object). 

As to claim 21 , VM teaches wherein the overlapping component classifiers 
comprises four corners representing a rear profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 
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As to claim 22, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a frontal profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 23, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a side profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 24, VM teaches wherein one of the overlapping component classifiers 
detects the bottom left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 25, VM teaches wherein one of the overlapping component 
classifiers detects the bottom right corner of a vehicle (page 461 , fig. 4 shows an object 
that's been identified and the system forming a box around the object, where the object 
is a vehicle driving on a road). 
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As to claim 26, VM teaches wherein one of the overlapping component classifiers 
detects the top left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 27, VM teaches wherein one of the overlapping component classifiers 
detects the top right corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 28, VM teaches wherein positioning of the four corners of the rear 
profile for a vehicle differ for different classes of vehicles (page 461 , fig. 4 shows an 
object that's been identified and the system forming a box around the object, where the 
object is a vehicle driving on a road). 

As to claim 29, VM teaches a class of vehicle includes sedans (car; page 459, 
left-hand column lines 12-14). 

As to claim 33, VM teaches wherein a class of vehicle includes trucks (truck; 
page 459, left-hand column lines 12-14). 
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As to claim 34, VM teaches wherein a distance between any two corners of the 
vehicle is constrained (page 461 , fig. 4 shows an object that's been identified and the 
system forming a box around the object, where the object is a vehicle driving on a road). 

As to claim 35, VM teaches wherein the constraint between any two corners of 
the vehicle is scaled based on a distance between the vehicle and a camera capturing 
the video sequence and camera parameters (page 459, left-hand column lines 18-21). 

As to claim 37, VM teaches wherein the accumulated confidence scores are 
inferred from confidence scores across multiple frames using a recursive filter (the 
algorithm used to determine the next trajectory adjusts itself whenever new information 
is received, thus making it repeat and re-occur; page 459, left-hand column lines 49-50; 
page 459, right-hand column lines 1-2). 

As to claim 38, VM teaches wherein when the accumulated confidence score is a 
linear combination of the confidence scores of multiple component classifiers and the 
whole-appearance classifiers (page 459, left-hand column lines 22-25). 

As to claim 39, VM teaches wherein when the confidence score for a principal 
component classifier is sufficiently high, the confidence score of the remaining 
component classifiers and the whole-appearance classifier are computed (page 459, 
left-hand column lines 15-21). 
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As to claim 40, VM teaches wherein if an object is detected the method 
comprising the step of: tracking the object over subsequent image frames (page 459, 
right-hand column lines13-20). 

As to claim 41 , VM teaches wherein the step of tracking the object further 
comprises the step of: restricting an area of search in each subsequent image frame 
based on the location of the object in a current image frame (page 459, right-hand 
column lines 24-31). 

As to claim 42, VM teaches wherein the step of tracking the object further 
comprises the step of: determining the optimal classifier scale based on a distance 
between the object and a camera detecting the object and camera parameters (page 
459, right-hand column lines 32-41 teaches the prediction process using speed, 
however, speed is the combination of time and distance, therefore there is the need to 
use the distance parameter). 

As to claim 45, VM teaches wherein an object class includes pedestrians 
(person; page 459, left-hand column lines 12-14). 

As to claim 47, VM teaches wherein an object class includes motorcycles 
(motorbike; page 459, left-hand column lines 12-14). 
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As to claims 49-51 , 53, 55, 58-80, 84-85 and 87-92, they are the system claims 
of claims 1 , 3, 45, 47, 2, 5-21 , 24-29, 34-35 and 37-42. Please see above for detail 
analysis. 

As to claim 57, VM teaches a display for displaying the video sequences (Fig. 4 
is a sample of a display window of the system). 

As to claim 105, it is the same as claim 1 with the exception of claim 105 does 
not teach wherein the component classifiers include classifiers for detection 
components at multiple scales, but in stead teaches wherein the one or more classifiers 
include overlapping components classifiers (Laumeyer teaches classifying the object 
detected based on color and shape ([0056], lines 14-18), since the two identifiers are for 
the same object they are considered to be overlapping). 
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5. Claims 30-32, 46, 48, 52, 54, and 81-83 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over VM. 

As to claims 30-32, 46 and 48, VM teaches the classification may have one of 
the following values: car, truck, motorbike, person, toojarge and mobile_object (page 
459, left-hand column lines 10-14). VM does not expressly disclose that the classifier 
also includes bicycles, sports utility vehicles, vans, tractor-trailers and traffic signs. 
However, Examiner takes Official Notice that the non-disclosed classifiers are well 
known in the art. It would have been obvious at the time of the invention was made to 
one of ordinary skill in the art to add the additional classifiers since Examiner takes 
official notice that the non-disclosed classifiers are common on a normal road. 

As to claims 52, 54, and 81-83, they are the system claims of claims 46, 48 and 
30-32. Please see above for detail analysis. 
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6. Claims 36, 43, 44, 56-57, 86 and 93-94 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over VM in view of Laumeyer, further in view of Brady et al. 
(5,761 ,326 hereinafter "Brady"). 

As to claim 36, VM does not teach wherein an image pyramid of multiple 
resolutions is used to detect objects of size 2*x, 4*x and so on with the classifier for the 
size x. Brady teaches a method for machine vision classification and tracking using 
multiple resolution of a region of interest that has a sample kernel of 2x2, even though 
this method only give one sample it is not limited to kernel size 2x2 (Col. 12, lines 34- 
45). Thus, Brady's machine vision method reads on the claimed pyramid multiple 
resolution method to detect objects. Therefore, it would have been obvious to one 
ordinarily skilled in the art at the time the invention was made to combine VM's 
surveillance system with Brady's multiple resolution system in order to increase the 
speed of processing the information for tracking (Brady Col. 12, lines 34-35). 

As to claim 43, VM does not teach wherein the confidence scores of component 
classifiers are computed in a coarse to fine framework. Brady teaches a method for 
machine vision classification and tracking that searches a target region first at a lower 
resolution then at a higher resolution (Col. 12, lines 43-46). Thus, Brady's searching a 
target region from low to high resolution reads on the claimed coarse to fine framework. 
Therefore, it would have been obvious to one ordinarily skilled in the art at the time the 
invention was made to combine VM's surveillance system with Brady's multiple 
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resolution system in order to increase the speed of processing the information for 
tracking (Brady Col. 12, lines 34-35). 

As to claim 44, VM does not teach wherein detection is performed on an image 
pyramid of multiple resolutions. Brady teaches a method for machine vision 
classification and tracking using multiresolution pyramid (Col. 12, lines 41-43). Thus, 
Brady's multiresolution pyramid reads on the claimed image pyramid of multiple 
resolutions. Therefore, it would have been obvious to one ordinarily skilled in the art at 
the time the invention was made to combine VM's surveillance system with Brady's 
multiple resolution system in order to increase the speed of processing the information 
for tracking (Brady Col. 12, lines 34-35). 

As to claim 56, VM does not teach wherein if an object is detected, the processor 
outputs a warning signal. Brady teaches the graphical user interface automatically 
displays alarm information when an incident has been detected (Col.4, lines 5-7). Thus, 
Brady's alarm information reads on the claimed warning signal. Therefore, it would 
have been obvious to one ordinarily skilled in the art at the time the invention was made 
to combine VM's surveillance system with the alarm notification in order to notify the 
user. 

As to claims 86 and 93-94, they are the system claims of claims 36 and 43-44. 
Please see above for detail analysis. 
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Conclusion 

7. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 



Application/Control Number: 10/805,946 Page 20 

Art Unit: 2624 

Contact Information 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CLAIRE WANG whose telephone number is (571)270- 
1051 . The examiner can normally be reached on M-F 9am-6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on 571-272-7778. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Matthew C Bella/ 

Supervisory Patent Examiner, Art 

Unit 2624 

Claire Wang 
02/12/2009 



